Using mathematical programming to solve Factored Markov Decision Processes with Imprecise Probabilities

AbstractThis paper investigates Factored Markov Decision Processes with Imprecise Probabilities (MDPIPs); that is, Factored Markov Decision Processes (MDPs) where transition probabilities are imprecisely specified. We derive efficient approximate solutions for Factored MDPIPs based on mathematical programming. To do this, we extend previous linear programming approaches for linear approximations in Factored MDPs, resulting in a multilinear formulation for robust “maximin” linear approximations in Factored MDPIPs. By exploiting the factored structure in MDPIPs we are able to demonstrate orders of magnitude reduction in solution time over standard exact non-factored approaches, in exchange for relatively low approximation errors, on a difficult class of benchmark problems with millions of states

Not available

Neste trabalho introduzimos uma maneira original de construir um Sistema Especialista: transformar uma Base de Conhecimento em um programa PROLOG. Enquanto que todos os Sistemas Especialista encontrados na bibliografia são controlados por um \"shell\", obtemos, com essa transformação, um Sistema Especialista cujo controle é feito pelo próprio PROLOG. Apresentamos também, um protótipo de um compilador para automatizar a referida transformação.This work introduces a way of transforming directly a Knowledge Base into a PROLOG program. With the method presented here we obtain an Expert System, the control of which is carried out by PROLOG itself, without requiring the use of a shell. We also present a model of a compiler which automates the mentioned method of transformation

Not available

Neste trabalho introduzimos uma maneira original de construir um Sistema Especialista: transformar uma Base de Conhecimento em um programa PROLOG. Enquanto que todos os Sistemas Especialista encontrados na bibliografia são controlados por um \"shell\", obtemos, com essa transformação, um Sistema Especialista cujo controle é feito pelo próprio PROLOG. Apresentamos também, um protótipo de um compilador para automatizar a referida transformação.This work introduces a way of transforming directly a Knowledge Base into a PROLOG program. With the method presented here we obtain an Expert System, the control of which is carried out by PROLOG itself, without requiring the use of a shell. We also present a model of a compiler which automates the mentioned method of transformation

Model based diagnosis for network communication faults

The lack of specialized professionals in network management and the growing complexity of this task has been aiming the need for developing tools to give support to the network administrator task. The construction of such tools requires an intense process of knowledge acquisition from experts in the area as well as the use of Artificial Intelligence (AI) techniques. A number of different approaches have been proposed, evolving from rule-based systems through case-based systems, to more recent modelbased systems [6] [7] [8] [9] [11]. A special attention has been given to propose systems to solve two main network management tasks: the fault diagnosis and performance management. The aim of this paper is to specify a Communication Fault Diagnostic System applying the AI Model Based approach.. We claim that this approach provides a foundation for exchanging behavioral, structural and control information between the sub-tasks of such complex systems. We also show what are the main aspects to be considered when constructing such systems: the construction of an automatic network discovery system and a configuration diagnosis system, both to support the construction of the network configuration model, and a network status gathering system to allow the diagnosis system to observe the network. 1

A logic-based agent that plans for extended reachability goals

Planning to reach a goal is an essential capability for rational agents. In general, a goal specifies a condition to be achieved at the end of the plan execution. In this article, we introduce nondeterministic planning for extended reachability goals (i.e., goals that also specify a condition to be preserved during the plan execution). We show that, when this kind of goal is considered, the temporal logic CTL turns out to be inadequate to formalize plan synthesis and plan validation algorithms. This is mainly due to the fact that the CTL`s semantics cannot discern among the various actions that produce state transitions. To overcome this limitation, we propose a new temporal logic called alpha-CTL. Then, based on this new logic, we implement a planner capable of synthesizing reliable plans for extended reachability goals, as a side effect of model checking

Efficient solutions to factored MDPs with imprecise transition probabilities

When modeling real-world decision-theoretic planning problems in the Markov Decision Process (MDP) framework, it is often impossible to obtain a completely accurate estimate of transition probabilities. For example, natural uncertainty arises in the transition specification due to elicitation of MDP transition models from an expert or estimation from data, or non-stationary transition distributions arising from insufficient state knowledge. In the interest of obtaining the most robust policy under transition uncertainty, the Markov Decision Process with Imprecise Transition Probabilities (MDP-IPs) has been introduced to model such scenarios. Unfortunately, while various solution algorithms exist for MDP-IPs, they often require external calls to optimization routines and thus can be extremely time-consuming in practice. To address this deficiency, we introduce the factored MDP-IP and propose efficient dynamic programming methods to exploit its structure. Noting that the key computational bottleneck in the solution of factored MDP-IPs is the need to repeatedly solve nonlinear constrained optimization problems, we show how to target approximation techniques to drastically reduce the computational overhead of the nonlinear solver while producing bounded, approximately optimal solutions. Our results show up to two orders of magnitude speedup in comparison to traditional "flat" dynamic programming approaches and up to an order of magnitude speedup over the extension of factored MDP approximate value iteration techniques to MDP-IPs while producing the lowest error of any approximation algorithm evaluated

A planner agent that tries its best in presence of nondeterminism

AbstractIn many nondeterministic planning domains, an agent whose goal is to achieve φ may not succeed in finding a policy that can guarantee that all paths from the initial state lead to a final state where φ holds (strong solution). Nevertheless, if the agent is trying its best to achieve φ, it cannot give up. Instead, it may be inclined to accept weaker guarantees, such as having a path leading to φ from any intermediate state reached by the policy (strong-cyclic solution), or even less, such as having at least one path leading to φ from the initial state (weak solution). But the agent should choose among such different options based on their availability in each situation. Although the specification of this type of goal has been addressed before, in this paper we show how a planner based on the branching time temporal logic α-ctl can be used to plan for intuitive and useful goals of the form “try your best to achieve φ”